1. Field of the Invention
The invention relates to remote direct memory access (RDMA) systems and, more specifically, to RDMA systems that support synchronization of distributed processes in a large scale multiprocessor system.
2. Description of the Related Art
Distributed processing involves multiple tasks on one or more computers interacting in some coordinated way to act as an “application”. For example, the distributed application may subdivide a problem into pieces or tasks, and it may dedicate specific computers to execute the specific pieces or tasks. The tasks will need to synchronize their activities on occasion so that they may operate as a coordinated whole.
In the art (e.g., message passing interface standard), “collective operations,” “barrier operations” and “reduction operations,” among others, have been used to facilitate synchronization or coordination among processes. These operations are typically performed in operating system library routines, and can require a large amount of involvement from the processor and kernel level software to perform. Details of the message passing interface can be found in “MPI—The Complete Reference”, 2nd edition, published by the MIT press, which is herein incorporated by reference.
Processes within an application generally need to share data with one another. RDMA techniques have been proposed in which one computer may directly transfer data from its memory into the memory system of another computer. These RDMA techniques off-load much of the processing from the operating system software to the RDMA network interface hardware (NICs). See Infiniband Architecture Specification, Vol. 1, copyright Oct. 24, 2000 by the Infiniband Trade Association. Processes running on a computer node may post commands to a command queue in memory, and the RDMA engine will retrieve and execute commands from the queue.